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ABSTRACT 

Monthly testing based rn a counterbalanced matrix 
sampling plan in one psychomotor, three affective , and five cognitive 
areas provides a continuous picture of the development of native 
American children enrolled in three distinct curricula. Of the nine 
scales, one shoved negative, two shoved insignificant, and six shoved 
significant positive changes, when compared vith national normat the 
results suggest that all three curricula have strong remedial effects 
across a broad raixge of important areas of preschool learning* 
(Author) 
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CONTINUOUS PROGRESS EVALUATION OF AMERICAN INDIAN PRESCHOOLERS 



Monthly testing based on a counterbalanced matrix 
sampling plan in one psychomotor, three affective, 
and five cognitive areas provides a continuous picture 
of the development of American Indian children enrol- 
led in three distinct curricula. Of the nine scales, 
one showed negative, two showed insignificant, and 
six showed positive changes. When compared with na- 
tional norms, the results suggest that all three cur- 
ricula have strong remedial effects across a broad 
range of important areas of preschool learning. 



The Los Angeles Indian Community 

The 25,000 Native Americans in Los Angeles^ outnumber all but the three 
or four largest tribes in the United States and represent an urban con- 
centration more than twice as large as that in any other city in the country. 
Most are members of families supported by a semi-skilled or unskilled heads 
of housenold. Approximately 10% are unemployed and a similar number are 
enrolled in trade schools or in one of the colleges or universities of the 
area. The adult population is relatively young, and their children are, 
for the most part, of preschool ana elementary school age (Kuchs & Havig- 
hurst, 1972). 

The majority of these American IndiaiiS maintain their traditional cuUures 
and make frequent visits to their reservations. Accustomed to small kinship 
groups, living in low-rent housing associated with high crime rates, mistrusting 
non-Indians, educationaPy deprived^ and politically powerless, urban Indians 
tend to have limited but intense relationships with a small circle' of 
relative and friends. They have difficulty relating to and obtaining services 



This figure from the 1970 census probably seriously underastimates tho 
actual Indian population of Los Angeles. Peter Snyder of the UCLA Anthro- 
pology Department uses a figure of 75,000 while C. L. White, et al , estimate 
in Regional Geography of Anglo AmeHcan (1974) that 100,000 Indians live in 
Los Angeles. 



from municipal, state, and federal agencies in the citieso Traditionally, 

the Bureau of Indian Affairs has worked hard to relocate Indians in the cities 

and then has refused to provide them with any services. 

Under the leadership of Sandy Gibbs, a Creek and Sac-Fox, Tribal American 

Consulting Corporation was formed in 1971 to develop programs to meet some 

of the Needs of the Los Angeles Indian community. Preschool education was 

selected as the priority concern. The Corporation directors felt that this 

was the most strategic point for improving Native American education, that 

American Indian preschool children in Los Angeles were not served by existing 
2 

programs, and that chances for significant success in this area were higher 
than with other possible programs. The Corporation was awarded a one-year 
grant by the State of California to operate a preschool for 90 children 
beginning in 1972. The school is now well into its third year, and has 
expanded to include a daycare center and two additional preschool s with a 
total enrollment of approximately 175. 

Preschool Programs for Indian Children 

The basic purpose of the preschool program is to enable Indian children 
to succeed in public school. Typically, Indians drop out of school by the 
fifth grade. Many recommendations have been made as to how to improve Indian 
education, but no data consistently support any particular set of ideas. 
In this situation it seemed appropriate to design a school with a number of 
alternative models of instruction. Children from a single community and meet- 
ing the requirements for enrollment in the compensatory education program 
represented a fairly consistent socioeconomic group. Randomly assigned to 
various instructional models, they would be the source of information useful 



In 1974, only six American Indians were enrolled in presCiool programs 
operated by the Los Angeles County Unified School Districtc 
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in selecting the best approach to preschool education for. urban Indian 
children. This new approach, of course, might turn out to be a combination 
of the instructional models that would be tried. 

To this end, the school staff set out to establish three distinct in- 
structional models. Common to all three classes would be a ratio of seven 
students to teacher or aide and a set of goals appropriate to the overall 
purpose of preparing the children for public school. However, the processes 
by which these objectives were to be pursued would vary significantly; each 
class would have either a Montessori, a traditional, or a cultural instruc- 
tional model . 

Developed originally for disadvantaged children in Italy in the 1880's 
and widely accepted as educationally sound, the Montessori model was a logi- 
cal choice for one of the three classrooms. This approach emphasizes the 
child's development through interaction with didactic materials related to 
a preplanned sequence of objectives. The goals of the Montessori classroom 
include development of independence, self-control, inner discipline, and a 
loving image of oneself. The materials cover the areas of "practical life," 
"sensorial experience," mathematics, langjage, science, geography, and music. 

The traditional model emphasizes interactions among children and their 
social, emotional, and cognitive development. Each day begins with group 
instruction by the teacher, after which the children work individually or 
in small groups under the direction of the teacher or aides. Materials are 
arranged in learning areas for mathematics, art, block play, reading, science, 
and socio-dramatic play. Curriculum planning stresses development of logical 
thought through tasks of classification, seriation, temporal relations, and 
spatial relations in Piagetian terms. 
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The major emphasis in the cultural model is on the environmental and 
cultural materials consistent with Indian cultures. The curriculum is plan- 
ned around six major units, each based on an ^'Indian culture area": the 
southwest, the northwest coast, the eastern woodlands, the plains, the great 
basin and California, and the "ancients" of Mexico and the southwest. Logical 
tasks, science, social studies, cooking, music, and art activities are pro- 
vided for each of the areas. Activities require cooperative effort: children 
work together to complete class projects in a noncompetitive environment 
modeled on many Indian cultures. This model explores the possibility of 
achieving the objectives of early education in a manner that is consistent 
with and enhances the child's pride in American Indian culture and traditions. 

The three instructional models have been evolving slowly since the 
school was established in 1972. The traditional model, least spectacular of 
the three, has also been steadiest. The two members of the teaching staff 
who remain from the first year are assigned to *^^is model. The class is 
quiet and relaxed and has proven particularly appropriate for the younger 
children in the school • 

The availability of teachers specifically trained in the Montessori 
model should have ensured that this approach would have been the easiest of 
the three to implement. However, the model requires a great amount of 
specialized and relatively expensive equipment that. is only gradually ac- 
quired in the real world of limited budgets. More interesting, particularly 
in light of Maria Montessori *s intention to develop a model appropriate to 
the disadvantaged, has been the difficulty of finding Montessori -trained 
teachers who could overcome a middle-class backgroud to function effectively 
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in the Indian communityo jince 1972> the Montessori model has been used 
for the instructional phases of daycare center activities. Children en- 
rolled in this modelremain in school all day, and represent a much wider 
Vange than do those in the other two models. 

The cultural model proved the most difficult to establish. It was not 
adequately conceptualized until the end of the first year. and teachers 
capable of developing the curriculum and working together were not found 
till the middle of the second year. The remainder of the second year w<»s 
one of ad hoc development but a grant from the Lily Foundation funded suiraner 
work and the third year began with a completed syllabus for the cultural model 
(Hall, et al., 1974). This class, which proved too active and unsettling 
for the younger children; has the highest mean age of children enrolled in 
the school. Thus, the needs of the children and the community led to abandon- 
ment of random assignment to the three models. In addition stat^ laws 
changed the original conception of the school. The grant to'develop programs 
for Indian children faced the problem of a state law that prohibited select- 
ing children on the basis of race, with the result approximately equal numbers 
of White and Mexican-American children have formed a significant minority 
of the school's enrollment since its inception. 
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Class 


N 


Mean Age in Janu- 


SD Age 


Percent 


Percent 






ary (in months) 


Indians 


Mai es 


Montessori 


45 


- 47.75 


8.09 


55 


48 


Traditional 


32 


42.03 


6.28 


62 


34 


Cultural 


34 


53.90 


3.11 


68 


38 


All Classes 


111 


48.00 


7.84 


61 


41 



TABLE 1 

Purpose of the Evaluation 

Evaluation has been a part of the program since inception of the school; 
since the second year the evaluation has been designed to achieve four pur- 
poses: 

1. To provide information to school administrators and staff useful 
in improving the educational program. 

2. To provide an independent judgment of the extent to which the 
program is achieving intended outcomes. 

3. To train American Indians in the basic techniques o^^ educational 
evaluation. 

4. To provide a laboratory setting for trying out ideas related to 
formative evaluation. 

Instruments 

Four tests were used to collect the basic evaluation information. Tests 
were chosen first on the relative importance of having information on an ob- 
jective, second on the basis of the quality of the instrument as indicated in 
the Seventh Mental MeCisuremente Yearbook (Buros, 1972) and the CSE-ECRS Pre- 
echooU Kindergarten y est Evaluations (Hoepfner^ et al., 1971 )« 

The first of the four tests is the 150-item Peabody Picture Vocabulary Test 
(PPV). The examiner shows the child four pictures printed on a single page 
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The figures are characteristic but only approximate, as dropouts and new en- 
rollees ensure constant fluctuation. Children in the Montessori model attend 
school all day, while only one-half those in the Traditional or Cultural models 
are in school at one time. g 
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a:=u a:>ks the child to indicate the one named by the examiner. For example, 
a child may be shown a line drawing of a crib, a woman, a kitten, and a. spoon, 
and asked to indicate the picture of the crib. 

The second test is the 64-item Cooperative Preschool Inventory (CPI). 
Three of the four scales are used. The Social Responsiveness score, inter- 
preted as a measure of self-concept, is based on the ability of the child 
to respond to questions such as "What is your first name?" The Associative 
Vocabulary score is based on questions such as "If vou want gas for your 
car, where would you go?" The Concept Activation-Sensory score, interpreted 
as a measu^-e of motor development, is based on judgments of quality of the 
child's response to directions such as "Copy this (triangle) for me." 

The third test (Tribal American Premathematics or TAP) is a locally 
developed 134-item premathematics instruments that measures counting ability, 
knowledge of the mathematical properties of objects such as size, weight 
and length, and knowledge of money and time. 

The fourth test used during 1973-1974 was the Primary Academic Sentiment 
Scale (PASS). In this test, the child marks one of three pictures to indicate 
which of three activities he would most like to do. For example, he may 
be asked whether he would most like to play ball, sit on his mother's lap, 
or go to a museum. Choice of the museum would count toward "motivation 
to learn" and choice of his mother toward "dependency"; choice of the ball 
would not be scored. 

Collection of Data 

The plan for data collection using these tests has three distinctive 
features. First, testing is done individually. That is, the examiner takes 



ERIC 



8 



a single child from his class to a special testing area (actually a quiet 
corner of the daycare center) and administers the test cn a one-to-one basis. 
There are two reasons for this approach. First* group testing of preschool 
and early elementary school -age children does not work well (Jone$, 1970; 
Kennedy, 1970; Klein, 1971), Second, it provides more thorough training in 
evaluation for the American Indians who are doing this work. 

The second important feature is monthly, rather than pre-and posttesting. 
Each child is tested eight times a year during the first week of the month 
between October and May* 

The third important feature Is a counterbalanced matri?( sampling plan. 
Children from each of the three classrooms were assigned randomly to one of 
'four groups. All the children In a group receive the same test each month 
and all tests in rotation during the school year. No two groups receive the 
same test In the same month. Combined with the monthly testing feature, 
this ensures data on student progress on a continuous basis without sub- 
jecting any one child to more than about 15 minutes of testing each month. 

4 

Results are portrayed graphically in Figures 1-7. Analysis of variance, 
with three levels of instruction and eight levels of time as factors, was 
originally conceived as appropriate to the situation. However, departure 
from random assignment of students to models and the scope of the dropout 
problem suggests age and total months enrol l(nent as covariates. Small sample 

1 

PASS results have been omitted, and this instrument is no longer bein^ used 

In the evaluation. Basically, results from all three models fall along the 
publishers fourth stanine for the dependency scale and within the "normal range" 
for the motivation scale. The slope is slightly positive for the dependency 
scale and slightly negative for the sentiment scale. The converging results 
suggest that criticism of the interdependence of the two scales is warranted. 
This leaves the resiilts essaatlally uninterpretable. 
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size, the confounding of treatment and teacher effects, and the constantly 
changing nature of the curriculum while these data were beinji collected suqqest 
that the effort and expense of such an analysis is not yet worthwhile. 

Key to Figures 1 7 
IHli Mean score* Hontessori Classroom 

Mean score, Traditional Classroom 
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Mean score. Cultural Classroom 

Mean score, all classrooms 

Publisher's norm. Where one nom is 
shown, it is for a 48 month old 
child. Where two are shown, the 
higher score is the norm for a 48. 
and the lower score is the norm 
for a 36 month old child. 




Oct Nov Dec Jan Feb Mar 

PeaDody Picture Vocabulary Test (PPV^ 

Figure 1 
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May 



n 




Counting 

Tribal American Premathematics (TAP) 
Figure 2 




Time and Money 
Tribal American Premathematics (TAP) 
Figure 3 
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Oct Nov Dec Jan Feb Mar Apr May 

Concepts 

Tribal American Premathematics (TAP) 
Figure 4 
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Associative Vocabulary 
Cooperative Preschool Inventory (CPI) 

Figure 5 
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Oct Nov Dec Jan Feb Mar Apr May 

RssDonsiveness 



Cooperative Preschool Inventory (CP!) 
Figure 7 
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School-wide averages on five of the ^even scales show a distinct drop 
in the last one or two months of the school year. This brings to mind the 
complaints of general exhaustion that one often hears from teachers and stu- 
dents toward the end of the school year and here these effects are found in a 
preschool! Of the two scales that do not show the effect, one (Figure 6) 
is a measure of motor development and the other (Figure 4) is the locally 
developed mathematics concepts test— the most dubious scale in the battery. 
If this pattern is universal, it may explain many evaluation results which 
have relied only on pre- and posttest measures. Whether statistically 
significant results are obtained could easily be a matter of timing. Know- 
ing that such a pattern is likely to exist permits design of an evaluation 
that will praise or condemn a program independent of actual program achieve- 
ment and dependent only on what the evaluator wants to prove! 

School -wide averages show little change between September and May on 
two of the seven scales. In one case (Figure 5), the results are approximately 
at the national norm for 48 month-old children for the entire year, except 
for March and April, when they are well above the normo It would appear 
from this that the school is having little effect on "associative vocabulary," 
in reality a measure of general knowledge about the community The other 
scale showing little cha'nge during the year (Figure 4) is based on an in- 
strument of low reliability that has since been revised. 

School-wide averages on Ihe remaining five instruments have approximately 
the same shape. That is, the slope tends to be zero or slightly positive for 
the first two or three months and is strongly positive from November to 
April. Gains are steadiest on the psychomotor scale (Figure 6) and greatest 
in vocabulary development (Figure 1) and counting skills (Figure 2). 

15 
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Children in the cultural model consistently outperform those in the 
Montessori model, and those in the Montessori model consistently outperform 
those in the traditional model* The most likely explanation of the differ- 
ences lies in the age of the children (Table 1), As the curves for all models 
are similar, all models seem equally efficient in achieving the objectives 
measured. However, student progress in the cultural model seems to accelerate 
significantly after the niid-yearo This acceleration coincides with the 
actual introduction of the cultural model which had existed in name only 
until December* 

Discussion 

Data from 1974-1975, will be more useful in determining the relative 
efficiency of the three models. During this school year all programs are 
operating as originally conceptualized. These data will even overcome to 
some extent the problem of nesting treatments and teachers; included in 
the data are a second school, opened in October 1974 which is following the 
cultural model; and a third school, opened in December 1974, which is 
following the traditional model. A real answer of course requires a still 
larger number of schools, and it is hoped that this will be possible begin- 
ning in September 1975. 

Should all models be equally efficient in achieving program objectives 
(as suggested), than a choice shoulc^ be made among them on the basis of 
either cost or affect. Selecting the cheapest program that will achieve 
desired objectives will permit operation of a greater number of schools 
and the education of a greater number of children. Selecting the program 
most desired by children and parents will increase the likelihood that 
Indian children will be enrolled in preschools. As there is little to 
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choose from on the basis of relative costs (salaries, rent, and food which 
account for about 90% of the budget, are about the same in all models), 
and as children cannnt be educated unlass present, the latter criteria is 
probably the more appropriate, if a choice must be made* 

Tribal American Consulting Corporation has consistently devoted about 
four percent of the program budget to evaluation and may be expected to 
continue to do so. A larger percentage would have detracted from the pro- 
gram per se and would not have been used efficiently, while the continuing 
expansion of the program has permitted continued improvement of the evalua- 
tion itself. The preschool evaluation provided an opportunity to experiment 
ith a number of techniques and ideas potentially useful in conducting evalua- 
tions* The Ideas themselves are not new, but comparing all of them in one 
program provided a long-term look at the way they work in practice and the 
way they interact with one another. Among these techniques and idears were 
(1) monthly testing, (2) adapting published tests to evaluation studies by 
using only needed subscales, (3) developing criterion-referenced tests lo- 
cally, (4) focusing the evaluation by development of questions from a 
general checklist of program characteristics, (5) selecting instruments ap- 
propriate to a particular situation, (6) sampling to increase the infor- 
mation obtained, (7) using graphs and modified del phi techniques for de- 
cision making, (8) estimating time and financial requirements for various 
phases of the evaluation, (9) reporting results, (10) developing working 
relationships with teachers, parents, and students, and (11) testing with 
a paraprofessional staff. 

Much of what was learned was incorporated in the CSE Formative Evalua- 
tion K it (Churchman, Petrosko, Spooner-Smith, 1974, 1975). This Kit is 
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designed for the typical elementary -school situation, in which programs 
must be developed without external evaluators, random assignments, control 
groups, special evaluation instruments, statistical tests, and without the other 
methods of the social scientist, but with a great deal of community and 
staff interest, resistance, or pressure. The preschool evaluation has led 
to the development of evaluation techniques tii-at are based on adapting 
social science methods to the school situation, rather than trying to force 
the school to adapt to the methods of the social scientist. If this Kit, 
now in national field test, proves in any way successful, the importance 
of the preschool evaluation lies not only in its contributions to develop- 
ment of the school program but also in its contributions to the develop- 
ment of evaluation methodologies. 
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